fix(server): regenerate an empty persisted secret instead of failing startup by juliusmarminge · Pull Request #32053 · pingdotgg/t3code

juliusmarminge · 2026-11-06T14:40:38Z

Problem

When <state>/userdata/secrets/server-signing-key.bin is 0 bytes, the server exits with code 1 on every start: CryptoError: Mac.key: An HMAC key cannot be empty. The desktop app retries forever. ServerSecretStore.getOrCreateRandom returns any existing secret file without checking its length. Signing now goes through Mac.key, which rejects an empty key, so the empty file is fatal. The same happens for any other secret loaded through getOrCreateRandom (for example mcp-oauth-signing-key.bin).

Reproduced on main 6577c6b: start the server once, stop it, truncate server-signing-key.bin to 0 bytes, start it again.

Change

getOrCreateRandom treats an empty persisted secret as missing: it removes the file and creates a fresh random secret through the existing atomic create path (link, with the AlreadyExists re-read). Two callers in one process can both see the empty file (PayloadSigner.make loads its key lazily), so the removal runs under a store-level permit after a second read and only removes a file that is still empty. That section is uninterruptible, so an interrupted caller keeps the permit until its removal has settled. Non-empty secrets and missing files take the same path as before.

Existing sessions signed with the old key need a fresh sign-in after the repair.

Scope and approval

Fixes #32026

Verification

Focused tests (vp test run src/auth/ServerSecretStore.test.ts src/auth/SessionStore.test.ts src/auth/McpOAuth.test.ts in apps/server): 3 files, 37 tests passed.

replaces an empty persisted secret with a fresh random one: fails on main with expected +0 to equal 32, passes here.
keeps the repaired secret when concurrent callers find it empty: two callers read the empty secret, and the second removal waits until the first caller has returned. Without the permit and re-read, the two callers return different keys; passes here.

vp run check:changed: format, lint, knip, graphql, typecheck (3 packages) and tests pass.

Runtime transcript, desktop mode, fresh base directory per side (synthetic state; driver below):

Before, main 6577c6b:

$ git -C <checkout> rev-parse --short HEAD
6577c6b618
$ t3 serve --mode desktop --base-dir <fresh>   # first start
INFO (#904): Listening on http://127.0.0.1:14397
(stopped pid 1829541)
$ truncate -s 0 <base-dir>/userdata/secrets/server-signing-key.bin
<base-dir>/userdata/secrets/server-signing-key.bin 0 bytes
$ t3 serve --mode desktop --base-dir <same>    # second start
server exited with code 1
[14:03:54.443] ERROR (#2): CryptoError: Mac.key: An HMAC key cannot be empty
    at fail (<checkout>/node_modules/.pnpm/effect@4.1.2/node_modules/effect/dist/crypto/internal/webCrypto.js:8:61)
    at PayloadSigner.fromSecret (apps/server/src/auth/SessionStore.ts:498:39)
    at PayloadSigner.fromSecret (definition) (apps/server/src/auth/PayloadSigner.ts:35:34)
<base-dir>/userdata/secrets/server-signing-key.bin 0 bytes

After, fa43bdc:

$ git -C <checkout> rev-parse --short HEAD
fa43bdc5aa
$ t3 serve --mode desktop --base-dir <fresh>   # first start
INFO (#902): Listening on http://127.0.0.1:14397
(stopped pid 2008917)
$ truncate -s 0 <base-dir>/userdata/secrets/server-signing-key.bin
<base-dir>/userdata/secrets/server-signing-key.bin 0 bytes
$ t3 serve --mode desktop --base-dir <same>    # second start
INFO (#573): Listening on http://127.0.0.1:14397
(stopped pid 2010275)
<base-dir>/userdata/secrets/server-signing-key.bin 32 bytes

Driver script

#!/usr/bin/env bash
# usage: run.sh <checkout> <label> <port>
set -u
CHECKOUT=$1; LABEL=$2; PORT=$3
HOME_DIR=/tmp/issue-lead-32026-r1/transcript/home-$LABEL-$(date +%s)
cd "$CHECKOUT/apps/server"
echo "\$ git -C $CHECKOUT rev-parse --short HEAD"; git -C "$CHECKOUT" rev-parse --short HEAD
start() {
  node src/bin.ts serve --mode desktop --base-dir "$HOME_DIR" --port "$PORT" > "$HOME_DIR.$1.log" 2>&1 & PID=$!
  for i in $(seq 1 60); do grep -q "Listening on" "$HOME_DIR.$1.log" && break; kill -0 $PID 2>/dev/null || break; sleep 1; done
  if kill -0 $PID 2>/dev/null; then grep -m1 -o "INFO.*Listening on.*" "$HOME_DIR.$1.log"; kill -TERM $PID; wait $PID 2>/dev/null; echo "(stopped pid $PID)"
  else wait $PID; echo "server exited with code $?"; grep -m1 -A1 "ERROR" "$HOME_DIR.$1.log" | sed 's#file://[^ )]*/apps/server/#apps/server/#'; grep -m2 "at PayloadSigner.fromSecret\|at server.runServerCommand" "$HOME_DIR.$1.log" | sed 's#file://[^ )]*/apps/server/#apps/server/#'; fi
}
echo "\$ t3 serve --mode desktop --base-dir <fresh>   # first start"; start boot1
echo "\$ truncate -s 0 <base-dir>/userdata/secrets/server-signing-key.bin"; truncate -s 0 "$HOME_DIR/userdata/secrets/server-signing-key.bin"; stat -c '%n %s bytes' "$HOME_DIR/userdata/secrets/server-signing-key.bin" | sed "s#$HOME_DIR#<base-dir>#"
echo "\$ t3 serve --mode desktop --base-dir <same>    # second start"; start boot2
stat -c '%n %s bytes' "$HOME_DIR/userdata/secrets/server-signing-key.bin" | sed "s#$HOME_DIR#<base-dir>#"

No UI changes, so there are no captures. Not checked: macOS and Windows file systems (the change uses the same readFile, remove and link calls as before).

AI-assisted; verified with the tests and tools listed above.

These tests reproduce #32026: getOrCreateRandom returns a 0-byte secret file as the key instead of replacing it with a fresh random secret. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…artup A 0-byte secret file under userdata/secrets was returned as-is by getOrCreateRandom, so an empty server-signing-key.bin became an empty HMAC key and the server exited on every start. getOrCreateRandom now removes an empty persisted secret and runs the existing generate-and-create path, so concurrent creators still converge through link's AlreadyExists. Fixes #32026 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… one key Two callers that both read an empty secret could each remove and recreate it, leaving the first caller holding a key that is no longer on disk. PayloadSigner.make loads its key lazily, so concurrent first signatures hit this in one process. Re-read under a store-level permit and remove only a file that is still empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Interrupting a caller released the permit while Node's removal was still in flight, so it could delete a key another caller had just repaired. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

macroscopeapp · 2026-11-06T14:42:35Z

Approvability

Verdict: Approved at fa43bdc

Macroscope's review found this PR approvable — Empty persisted secrets are regenerated instead of preventing startup, while valid secrets remain unchanged. Repair uses serialized rereading and the existing secure generation and persistence safeguards.

No code changes detected at 00c9bcc. Prior analysis still applies.

^{You can add or adjust custom eligibility rules. Learn more.}

github-actions · 2026-11-06T14:44:39Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	5.0 KiB	5.0 KiB	0 B (0.0%)	6.8 KiB	✅
Codex	Thread snapshot wire	3.8 KiB	3.8 KiB	0 B (0.0%)	4.9 KiB	✅
Codex	Live turn WebSocket wire	1.2 KiB	1.2 KiB	0 B (0.0%)	2.0 KiB	✅
Codex	Live turn WebSocket decoded	21.5 KiB	21.5 KiB	0 B (0.0%)	24.4 KiB	✅
Codex	Live turn messages	1	1	0 (0.0%)	8	✅
Claude	Total thread wire	5.0 KiB	5.0 KiB	0 B (0.0%)	6.8 KiB	✅
Claude	Thread snapshot wire	3.8 KiB	3.8 KiB	0 B (0.0%)	4.9 KiB	✅
Claude	Live turn WebSocket wire	1.2 KiB	1.2 KiB	0 B (0.0%)	2.0 KiB	✅
Claude	Live turn WebSocket decoded	21.9 KiB	21.9 KiB	0 B (0.0%)	24.4 KiB	✅
Claude	Live turn messages	1	1	0 (0.0%)	8	✅

Baseline: d290c1f · PR result: 00c9bcc · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 108.5 KiB
Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

coderabbitai · 2026-11-06T14:46:30Z

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

Source mode: Auto
Repository: pingdotgg/t3code
Branch: fix/issue-32026
File: .coderabbit.config.ts
Commit: 00c9bcc
Layers: Repository TS config
Review profile: CHILL
Plan: Team
Run ID: 5f4aceb2-ca65-4e0a-9f8a-741b6924c101

Commits

Reviewing files that changed from the base of the PR and between fa43bdc and 00c9bcc.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

Trigger review

No actionable comments were generated in the recent review.

Walkthrough

getOrCreateRandom now rechecks empty persisted secrets under a semaphore and removes them if they remain empty. Tests cover replacement with a 32-byte secret and concurrent callers.

Changes: Empty Secret Recovery

Layer / File(s)	Summary
Semaphore-protected repair and tests `apps/server/src/auth/ServerSecretStore.ts`, `apps/server/src/auth/ServerSecretStore.test.ts`	`getOrCreateRandom` re-reads an empty secret under a one-permit semaphore and removes it if it is still empty. Tests verify a 32-byte replacement and matching results from concurrent callers.

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: t3dotgg

Merge Risk: Minimal · previous review fa43b

The empty-secret repair is mergeable after normal checks; no concrete blocking issue remains.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title clearly describes the empty-secret repair.
Description check		The description covers the problem, fix, linked issue, and verification. It does not identify the AI model and harness or confirm that the linked issue was triaged.
Linked Issues check		[`#32026`] `getOrCreateRandom` replaces an empty persisted secret through the existing atomic creation path. Tests cover replacement, persistence, and concurrent repair. The reported desktop test shows …
Out of Scope Changes check		[`#32026`] The secret-store change and its tests support empty-secret repair and safe concurrent calls. The summary shows no unrelated changes.

Full details: Linked Issues check

Explanation

[#32026] getOrCreateRandom replaces an empty persisted secret through the existing atomic creation path. Tests cover replacement, persistence, and concurrent repair. The reported desktop test shows the server starts with a regenerated key.

Finishing Touches

Generate docstrings

Commit to this branch
Create a new PR

Generate unit tests (beta)

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

Source mode: Auto

Repository: pingdotgg/t3code

Branch: fix/issue-32026

File: .coderabbit.config.ts

Commit: fa43bdc

Layers: Repository TS config

Review profile: CHILL

Plan: Team

Run ID: `183380b6-7e4e-4051-8c77-48788fe3c1e7`

Commits

Reviewing files that changed from the base of the PR and between `6577c6b` and `fa43bdc`.

Files selected for processing: 2

`apps/server/src/auth/ServerSecretStore.test.ts`

`apps/server/src/auth/ServerSecretStore.ts`

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.

juliusmarminge · 2026-11-10T21:33:36Z

Evidence

Ran separately built real servers on Linux x86_64 with Node v26.12.1, fresh disposable state, and T3CODE_DEV_SEED=0. Both recordings use the same 1600×1080 terminal presentation; credentials and local paths are omitted.

Same steps on both sides: start and stop to generate secrets; restart with the intact key and verify byte-for-byte preservation; stop; truncate userdata/secrets/server-signing-key.bin to zero bytes; restart with the same state and inspect startup output and file size. Successful starts also returned HTTP 200 from /.well-known/t3/environment.

Before	After
d290c1f1b6	00c9bccdde
Server recording	Server recording
Empty-key restart exits 1: `Mac.key: An HMAC key cannot be empty`. File remains 0 bytes.	Empty-key restart reaches ready state and returns HTTP 200. File becomes a fresh 32-byte key.
Existing intact key remains unchanged.	Existing intact key remains unchanged.
Transcript	Transcript

Reproduction driver · Recording renderer

AI-assisted; verified with the tests and tools listed above.

Not shown:

Electron's repeated startup-retry UI was not exercised; the underlying failure and recovery were demonstrated through the real built server.
Concurrent secret creation and other secret types were not exercised; this reproduction covers the persisted signing key.

juliusmarminge and others added 4 commits November 6, 2026 13:53

test(server): reproduce empty persisted secret reuse (#32026)

de10f2c

These tests reproduce #32026: getOrCreateRandom returns a 0-byte secret file as the key instead of replacing it with a fresh random secret. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fix(server): hold the repair permit until an interrupted removal settles

fa43bdc

Interrupting a caller released the permit while Node's removal was still in flight, so it could delete a key another caller had just repaired. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:S 10-29 changed lines (additions + deletions). labels Nov 6, 2026

macroscopeapp Bot approved these changes Nov 6, 2026

View reviewed changes

juliusmarminge added bugfix Fix PR opened by an issue lead; enters the PR loop pr:needs-julius Needs Julius: route gate, unverifiable surface, or blocked labels Nov 6, 2026

shivamhwp mentioned this pull request Nov 7, 2026

Server won't start when server-signing-key.bin is empty ("Mac.key: An HMAC key cannot be empty") #32026

Open

Merge remote-tracking branch 'origin/main' into fix/issue-32026

00c9bcc

Navigation Menu

Repository navigation

fix(server): regenerate an empty persisted secret instead of failing startup - #32053

fix(server): regenerate an empty persisted secret instead of failing startup#32053
juliusmarminge wants to merge 5 commits into
mainfrom
fix/issue-32026

juliusmarminge commented Nov 6, 2026

Uh oh!

macroscopeapp Bot commented Nov 6, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 6, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Nov 6, 2026 •

edited

Loading

Review skipped

Uh oh!

juliusmarminge commented Nov 10, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

juliusmarminge commented Nov 6, 2026

Problem

Change

Scope and approval

Verification

Uh oh!

macroscopeapp Bot commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

github-actions Bot commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Thread transfer impact

Uh oh!

coderabbitai Bot commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

juliusmarminge commented Nov 10, 2026

Evidence

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

macroscopeapp Bot commented Nov 6, 2026 •

edited

Loading

github-actions Bot commented Nov 6, 2026 •

edited

Loading

coderabbitai Bot commented Nov 6, 2026 •

edited

Loading